A generalized mean field algorithm for variational inference in exponential families
نویسندگان
چکیده
We present a class of generalized mean field (GMF) algorithms for approximate inference in exponential family graphical models which is analogous to the generalized belief prop agation (GBP) or cluster variational meth ods. While those methods are based on over lapping clusters, our approach is based on nonoverlapping clusters. Unlike the cluster variational methods, the approach is proved to converge to a globally consistent set of marginals and a lower bound on the likeli hood, while providing much of the flexibility associated with cluster variational methods. We present experiments that analyze the ef fect of different choices of clustering on infer ence quality, and compare GMF with belief propagation on several canonical models.
منابع مشابه
Variational inference in graphical models: The view from the marginal polytope
Underlying a variety of techniques for approximate inference—among them mean field, sum-product, and cluster variational methods—is a classical variational principle from statistical physics, which involves a “free energy” optimization problem over the set of all distributions. Working within the framework of exponential families, we describe an alternative view, in which the optimization takes...
متن کاملGraph Partition Strategies for Generalized Mean Field Inference
An autonomous variational inference algorithm for arbitrary graphical models requires the ability to optimize variational approximations over the space of model parameters as well as over the choice of tractable families used for the variational approximation. In this paper, we present a novel combination of graph partitioning algorithms with a generalized mean field (GMF) inference algorithm. ...
متن کاملVariational Learning : From exponential families to multilinear systems
This note aims to give a general overview of variational inference on graphical models. Starting with the need for the variational approach, we proceed to the derivation of the Variational Bayes EM algorithm that creates distributions on the hidden variables in a graphical model. This leads us to the Variational message Passing algorithm for conjugate exponential families, which is shown to res...
متن کاملGraphical Models, Exponential Families, and Variational Inference
The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimizati...
متن کامل